…ypes of devices, not only disks).

Signed-off-by: Konstantina Chremmou <konstantina.chremmou@cloud.com>

Signed-off-by: Konstantina Chremmou <konstantina.chremmou@cloud.com>

The syslog feature allows to run a program and redirect its output/error
stream to system log. It's triggered passing "syslog_stdout" argument to
"Forkhelpers" functions like "execute_command_get_output".
To test this feature use a small preload library to redirect syslog writing.
This allows to test program without changing it.
The log is redirected to /tmp/xyz instead of /dev/log. The name was chosen
to allow for future static build redirection.
The C program is used only for the test, so the code style take into account
this (specifically it does not try to handle all redirect situation and all
error paths).

Signed-off-by: Frediano Ziglio <frediano.ziglio@cloud.com>

Provide an API/CLI for multi-version drivers. Two new classes represent
them: Host_driver and Driver_variant. See

  * doc/content/toolstack/features/MVD/index.md

for technical background and feature overview.

The current implementation mocks the missing drivertool that manipulates
the file system and observes driver state. We expect to integrate this
at a later point. Consequently, the current implementation is incomplete
and exists to faciliate development of API and XE clients.

On Xapi start and restart driver information is purged from the
database and re-built from scrach. This could be improved in the future
to be incremental.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>

When scanning a host for drivers, don't remove and re-create them but
instead update an entry if it already exists.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>

When scanning a host for drivers, don't remove and re-create them but
instead update an entry if it already exists.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>

Address review comments; fix typos and parameter names.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>

This fixes a typo in list of fields we show in the CLI.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>

Re-generate datamodel_lifecycle.ml to reflect correctly the current
version of the API.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>

Co-authored-by: Pau Ruiz Safont <psafont@users.noreply.github.com>
Signed-off-by: Christian Lindig <christian.lindig@citrix.com>

Co-authored-by: Pau Ruiz Safont <psafont@users.noreply.github.com>
Signed-off-by: Christian Lindig <christian.lindig@citrix.com>

Co-authored-by: Pau Ruiz Safont <psafont@users.noreply.github.com>
Signed-off-by: Christian Lindig <christian.lindig@citrix.com>

Fix documentation for select() method.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>

Use more existing functions to format records.

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>

…e.ml

Signed-off-by: Christian Lindig <christian.lindig@cloud.com>

…y the API. (xapi-project#6219)

With regard to the error removal, these entries had been added to the
SDK to account for specialised error messages needed by XenCenter, which
was not the right thing to do; this should have been done on the
implementing client side, not the SDK.

    MVD CP-52334 multi-version driver API/CLI

Provide an API/CLI for multi-version drivers. Two new classes represent
    them: Host_driver and Driver_variant. See

      * doc/content/toolstack/features/MVD/index.md

    for technical background and feature overview.

The current implementation mocks the missing drivertool that manipulates
the file system and observes driver state. We expect to integrate this
at a later point. Consequently, the current implementation is incomplete
    and exists to faciliate development of API and XE clients.

    On Xapi start and restart driver information is purged from the
database and re-built from scrach. This could be improved in the future
    to be incremental.

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>

The previous feature processing logic was a bit ad hoc with various
random conversion functions. Abstract them into one module and refactor
a few functions to make their uses more explicit.

Also try to encourage users, when processing features, to convert them
into `Feature.t` and use the provided functions in this module rather
than writing `List.mem` directly.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>

This is so that `Storage_migrate` can use this function, otherwise there
would be a dependency cycle. This function also sounds like a utility
function.

Also add an mli file for `Storage_utils`.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>

This includes:
- VDI.compose for composing VDIs. Note the ordering of the parameter
  `parent` and `child` is reversed, due to the inconsistency between
  SMAPIv2 and SMAPIv3
- DP.attach_info which calls the idempotent DP.attach3 on a already
  attached VDI to get the attach info again.
- VDI.{add_to,remove_from}sm_config for xapi to keep track of the mirror
  vdi
- VDI.set_content_id for determining similarity between VDIs

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>

This API call prepares a nbd server that will be used for mirroring.
This gets mapped to different logic on SMAPIv1 and SMAPIv3.

- for SMAPIv1, this will reuse the old logic of xapi using Tapctl to
  retrieve this information from tapdisk
- for SMAPIv3, this uses the new `import_activate` call to prepare an
  nbd server

Also adjust the position of the module `DATA` to use the `DP` module.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>

Previously we hardcoded the export to be empty string when the nbd
client negotiates with the server. This patch allows specifying this
as a protocol parameter.

This is useful for SXM of SMAPIv3 SRs when we copy the snapshot from
source to destination, where the export name is needed when we copy
using nbd.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>

This is the main commit that implements the tracking logic of SXM for
inbound SMAPIv3 SRs. It mainly consists of the following changes:

- Adding parameters to various SMAPIv2 calls to track the domain slice
  that is invoking the SMAPI calls. For SXM, we track two particular
  domains: the mirror domain and the copy domain, corresponding to two
  of the operations happening during migration: mirror of the VDI and
  copy of the base image. Although this was not needed for SMAPIv1 calls,
  it is now necessary for SMAPIv3 calls since it requires explicit tracking
  of the domain slices.
- Extend various SMAPIv2 calls with the new domain parameter since
  SMAPIv3 has one qemu-dp process per domain, and therefore requires us
  to keep track of the domain parameter.
- Invoking the `import_activate` call in the new `nbd_handler` on the
  receiving end to prepare an nbd server for mirroring.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>

SXM using qemu datapaths is done in two parts: inbound and outbound,
hence the storage side has declared a `VDI_MIRROR_IN` features.
Implement the checking logic for this feature here. This is done in two
parts:

1. Check the relevant SM for the feature `VDI_MIRROR_IN`. This is done
   on the destination host. If this fails, then storage migration will
   be prevented from happening in the beginning. Consider this as a
   pre-check.
2. Check in xapi-storage-script that the chosen datapath supports
   mirroring. Technically mirroring is a datapath feature so it makes
   sense to check it there. But currently there is no easy way to
   register a datapath feature into the SM object in xapi database, so
   instead the volume plugins for SMAPIv3 SRs would also declare this
   feature. However, we still need a final check on the datapath
   features to make sure the selected datapath really supports this
   feature. Note this check happens while trying to set up the mirror,
   after storage migration has started, and will give a relatively
   cryptic error to the user.

The end result of this is that if either the volume plugin does not
declare `VDI_MIRROR_IN`/`VDI_MIRROR` or the dp plugin does not declare
any of these two, SXM will fail (upfront/mid way). Note current SXM of
SMAPIv1 SRs are not affected since `VDI_MIRROR` is a superset of
`VDI_MIRROR_IN`.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>

Signed-off-by: Mark Syms <mark.syms@cloud.com>

xapi-project#6245)

…le to XenServerTest only when specified.

Making the internals visible to a friend assembly works only if both
assemblies are signed with a strong name or both unsigned. The code
won't compile in scenarios where only the source code needs to be built
and signed with a .snk key, therefore I'm suggesting using a
preprocessor directive that can be switched on and off by the build
caller (the other option is to put both source and test projects in a
solution and build and sign both at all times, but at the moment I think
we can go with the easy option).

These were added for the SMAPIv3-inbound VM migrations

Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>

These were added for the SMAPIv3-inbound VM migrations

On all platforms tested the canonical location for dracut is `/usr/bin/dracut`.
Older platforms symlink to it from `/usr/sbin/dracut`, whilst newer ones do not,
thus on these platforms xapi is unable to find dracut and fails to start.

Adjust the path to the canonical location to ensure it operates correctly on all
platforms.

Also adjust a reference in network_utils.ml (whilst newer platforms also seem to
have the symlink from `/sbin/dracut`, it would seem to make sense to point to
the canonical location regardless).

Signed-off-by: Alex Brett <alex.brett@cloud.com>

Minor updates of the Hugo documentation:

- The current Ubuntu snap package of Hugo is not supported by the docs.
  We should take a first minor step towards fixing this.

- `doc/README.md` is outdated and should be updated.
  It says that the Ubuntu snap of Hugo works, but it does not anymore.
  Fix this by updating the outdated information.

- An initial fix is to update the Relearn theme from 5.20.x to 5.23.0:
  - It does not introduce breaking changes.
  - It introduces more straightforward page links and deprecates older syntax.
  - Fix the warnings by updating relative links accordingly.

Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>

On all platforms tested the canonical location for dracut is
`/usr/bin/dracut`. Older platforms symlink to it from
`/usr/sbin/dracut`, whilst newer ones do not, thus on these platforms
xapi is unable to find dracut and fails to start.

Adjust the path to the canonical location to ensure it operates
correctly on all platforms.

Also adjust a reference in network_utils.ml (whilst newer platforms also
seem to have the symlink from `/sbin/dracut`, it would seem to make
sense to point to the canonical location regardless).

Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>

Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>

…6251)

Minor updates of the Hugo documentation:

- The current Ubuntu `snap` package of Hugo is not supported by the
docs.
  We should take a first minor step towards fixing this.

-
[`doc/README.md`](https://github.com/xapi-project/xen-api/blob/master/doc/README.md)
is outdated and should be updated ([new
version](https://github.com/xenserver-next/xen-api/blob/docs-update-README-and-hugo-relearn-to-5.23/doc/README.md)).
  It says that the Ubuntu snap of Hugo works, but it does not anymore.
  Fix this by updating the outdated information.

- An initial fix is to update the Relearn theme from 5.20.x to 5.23.0:
  - It does not introduce breaking changes.
- It introduces more straightforward page links and deprecates older
syntax.
  - Fix the warnings by updating relative links accordingly.

A preview is available on my site:
https://xenserver-next.github.io/xen-api/index.html

Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>

…instead

Signed-off-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>

When the coordinator restarts. the no-other-masters check in the startup
sequence does two things for each pool member:

1. It checks that the host agrees that it is are not the coordinator.
2. It unblocks the host's master_connection thread, which is likely waiting
   for a reconnection delay to expire, which may be up to 256 seconds
   (exponential backoff is used). The delay is interrupted to
   immediately unblock DB calls.

Licensing initialisation comes earlier in the startup sequence, but
under certain circumstance make calls to other host, in particular after
an upgrade. A this time, hosts may still be blocked on the
master_connection for up to 256 s, which adds an unnecessary delay to
the coordinator's startup sequence and therefore the usability of the
API.

Address this by reversing the order of the two startup actions.

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

…instead (xapi-project#6255)

For links to internal pages, `{{site.baseurl}}` (without the leading
`.`) does not
work in current Hugo, and it is better to replace it by using relative
links:

`{{.Site.BaseURL}}` is replaced in the rendered output, but it doesn't
work when
viewing and editing the files using Markdown editors and viewers like on
GitHub/Lab
as well as IDEs with improved support for Markdown editing(in editor and
preview).

As the relative links work in all cases like in the editors and
previews,
fix those (currently broken) links to use the relative links instead.

While at those, also replace the link name "here" with a more specific
link name.

PS: The only place were `{{.Site.BaseURL}}` or similar is needed is the
link in
the `logo.html` template for
`img="{{.Site.BaseURL}}/images/xapi-project.png"`

…oject#6257)

When the coordinator restarts. the no-other-masters check in the startup
sequence does two things for each pool member:

1. It checks that the host agrees that it is are not the coordinator.
2. It unblocks the host's master_connection thread, which is likely
waiting
   for a reconnection delay to expire, which may be up to 256 seconds
   (exponential backoff is used). The delay is interrupted to
   immediately unblock DB calls.

Licensing initialisation comes earlier in the startup sequence, but
under certain circumstance make calls to other host, in particular after
an upgrade. A this time, hosts may still be blocked on the
master_connection for up to 256 s, which adds an unnecessary delay to
the coordinator's startup sequence and therefore the usability of the
API.

Address this by reversing the order of the two startup actions.

The call currently calls `network.attach` on all networks, but this
fails for PIFs that are not managed by xapi. Simply skip those.

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

…api-project#6259)

The call currently calls `network.attach` on all networks, but this
fails for PIFs that are not managed by xapi. Simply skip those.

This reverts commit 889dfa6. As there
is no need to clean up the source VM earlier at all, VM_save already
does the job of deactivating the source VM datapath.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>

Previously the `post_detach_hook` was run after the VDI is detached on
the source VM, which is at the very end of the SXM, where the source VM
is shutdown. However, the job of `post_detach_hook` is to call
`Remote.receive_finalize` which will destroy the mirroring datapath.
This should have been called as soon as we deactivate the datapath on
the source VM, at which point the source VM will stop writing using that
datapath. This commit changes `post_detach_hook` to
`post_deactivate_hook` and moves its calling locations accordingly.

Signed-off-by: Vincent Liu <shuntian.liu2@cloud.com>

…api-project#6254)

Optional improvement for fast CI completion when only Hugo docs change
on push/PRs:

When only Hugo docs change on push and PRs, other workflows can be
skipped using
`paths-ignore:` for `docs/**` and `.github/workflows/hugo.yml` to not
trigger other
workflows if only these paths change when pushing to a branch / work on
a PR.

This is completely optional, but would be nice to have when only working
on Hugo docs.
It does not change scheduled CI runs. It changes only on-demand push/PR
workflow runs.

…t_deactivate (xapi-project#6260)

There are two parts to this PR:

1. Reverts the VM migration ordering change, which was previously
thought to be necessary as I did not see `VM_save` would actually
deactivate the source VM datapath. Thanks @edwintorok for pointing that
out.
2. Change `post_detach_hook` to `post_deactivate_hook`, as this should
have been called as soon as the datapath is deactivated, at which point
all r/w using that datapath will be stopped.

More details in the commit message.

Signed-off-by: Konstantina Chremmou <konstantina.chremmou@cloud.com>

Signed-off-by: Edwin Török <edwin.torok@cloud.com>